Information apparatus for playing digital content that is received from a digital content service provided over the internet

ABSTRACT

An information apparatus (e.g., Internet appliance, information pad) and software applications of the information apparatus that includes digital camera or video camera and supporting voice commands for receiving and playing digital content from a service provided over a network (e.g., Internet) are herein disclosed and enabled. The mobile information apparatus may establish a wireless local area network connection to access a service over the Internet. The service may be a subscribed service. While connected to the service, the mobile information apparatus may capture, using the digital camera or video camera, digital content that includes the image or the video; send the captured the digital content to the service; and, thereafter receives, from the service, output data related to the digital content for outputting or playing at display device associated with the information apparatus.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/785,190, filed Oct. 16, 2017, which is a continuation of U.S. patentapplication Ser. No. 09/992,413 filed Nov. 18, 2001 and now issued asU.S. Pat. No. 9,965,233, which claims benefit of U.S. Provisional PatentApplication Ser. No. 60/252,682 filed Nov. 20, 2000. Additionally, U.S.patent application Ser. No. 15/785,190 is a continuation-in-part of U.S.patent application Ser. No. 10/053,765 filed Jan. 18, 2002 and nowissued as U.S. Pat. No. 9,836,257, which claims priority to U.S.Provisional Patent Application Ser. No. 60/262,764, filed Jan. 19, 2001.Moreover, U.S. patent application Ser. No. 15/785,190 is acontinuation-in-part of U.S. patent application Ser. No. 13/710,299filed Dec. 10, 2012 and now issued as U.S. Pat. No. 9,798,516, which isa continuation of U.S. patent application Ser. No. 12/903,048 filed Oct.12, 2010 and now issued as U.S. Pat. No. 8,332,521, which is acontinuation of U.S. patent application Ser. No. 10/016,223 filed Nov.1, 2001 and now issued as U.S. Pat. No. 7,941,541, and which claimsbenefit of U.S. Provisional Patent Application Ser. No. 60/245,101,filed Nov. 1, 2000. The complete disclosures of the above patentapplications are hereby incorporated by reference for all purposes.

TECHNICAL FIELD OF THE INVENTION

Present invention relates to providing digital content to an outputdevice and, in particular, to providing pervasive output in which aninformation apparatus can pervasively output digital content to anoutput device regardless of the processing power, display screen sizeand memory space of the information apparatus.

BACKGROUND AND SUMMARY OF THE INVENTION

As described herein, information apparatuses refer generally tocomputing devices, which include both stationary computers and mobilecomputing devices (pervasive devices). Examples of such informationapparatuses include, without limitation, desktop computers, laptopcomputers, networked computers, palmtop computers (hand-held computers),personal digital assistants (PDAs), Internet enabled mobile phones,smart phones, pagers, digital capturing devices (e.g., digital camerasand video cameras), Internet appliances, e-books, information pads, anddigital or web pads. An output device may include any one or more of faxmachines, printers, copiers, image and/or video display devices (e.g.,televisions, monitors and projectors), and audio output devices.

For simplicity and convenience, hereafter, the following description mayrefer to an output device as a printer and an output process asprinting. However, it should be understood that the term printer andprinting used in the discussion of present invention may refer to aspecific example used to simplify description or may be one exemplaryembodiment. The reference to printer and printing used here is intendedto be applied or extended to the larger scope and definition of outputdevices and should not be construed as restricting the scope andpractice of present invention.

Fueled by ever-increasing bandwidth and processing power andever-increasing numbers of wireless mobile devices and availablesoftware applications for pervasive devices, millions of users are orwill be creating, downloading, and transmitting content and informationusing their pervasive computing devices. As a result, there is a need toallow users to easily output content and information from theirpervasive computing devices to any output device. People need to outputdirectly and conveniently from their pervasive information apparatus,without depending on synchronizing with a stationary computer (e.g.,desktop personal computer) for printing, as an example.

To illustrate, an information worker at an airport receiving Email inhis hand-held computer may want to walk up to a nearby printer or faxmachine to have his e-mail printed. In addition, the mobile worker mayalso want to print a copy of his to-do list, appointment book, businesscard, and his flight schedule from his mobile device. As anotherexample, a user reading a news article using his/her Internet-enabledpager or mobile phone may want to print out the complete article insteadof reading it through the small screen on the pager or mobile device. Instill another example, a user visiting an e-commerce site using hismobile phone may want to print out pictures of the product he/she isbuying because the pictures (image and/or graphics) may not be displayedon the small screen of his/her mobile phone. In yet another example, auser who takes a picture with a digital camera may want to easily printit out to a nearby printer. In still another example, a user with amobile device may want to simply walk up to a printer and convenientlyprint a file that is stored on the mobile device or that is stored on anetwork (e.g., Internet, corporate network) and accessible from themobile device, such as a PowerPoint® display application document, wordprocessing document, or a document in any other file format such as PDF,HTML, JPEG etc. In addition, a user should also be able to print a webpage or a book or a report published on the Internet. Finally, a usermay want to output a complete web page or any document or file to alarger display screen nearby, even though the small screen of his/hermobile device cannot completely display or open a document of such asize.

Conventionally, an output device (e.g., a printer) is connected to aninformation apparatus via a wired connection such as a cable line. Awireless connection is also possible by using, for example, radiocommunication or infrared communication. Regardless of wired or wirelessconnection, a user must first install in the information apparatus anoutput device driver (e.g., printer driver in the case the output deviceis a printer) corresponding to a particular output device model andmake. Using a device-dependent or specific driver, the informationapparatus may process output content or digital document into a specificoutput device's input space (e.g., printer input space). The outputdevice input space corresponds to the type of input that an outputdevice (e.g., a printer) understands (herein referred to as output dataor print data in the case the output device is a printer). For example,the printer input space or print data may include printer specific inputformat (e.g., one or more of an image format, graphics format, textformat, audio format, video format, file format, and data format),encoding, language (e.g., page description language, markup languageetc), instructions, protocols or data that can be understood or used bya particular printer make and model.

Output data may be proprietary or published or a combination of the two.An output device's input space or output data is therefore, in general,device dependent. Different output device models may have their owninput spaces specified, designed or adopted by the output devicemanufacturer (e.g., the printer manufacturer) according to aspecification for optimal operation. Consequently, different outputdevices usually require use of specific output device drivers (e.g.,printer drivers) for accurate output (e.g., printing). For example, aprinter driver may control, manage, communicate, and output print datato a printer. Sometimes, instead of using a device driver (e.g., printerdriver), the device driving feature may be included or as part of anapplication software.

Installation of a device driver (e.g., printer driver) or applicationmay be accomplished by, for example, manual installation using a CD orfloppy disk supplied by the printer manufacturer. Or alternatively, auser may be able to download that particular driver or application froma network. For a home or office user, this installation process may takeanywhere from several minutes to several hours depending on the type ofdriver and user's sophistication level with computing devices andnetworks. Even with plug-and-play driver installation, the user is stillrequired to execute a multi-step process for each printer or outputdevice.

This installation and configuration process adds a degree of complexityand work to end-users who may otherwise spend their time doing otherproductive or enjoyable work. Moreover, many unsophisticated users maybe discouraged from adding new peripherals (e.g., printers, scanners,etc.) to their home computers or networks to avoid the inconvenience ofinstallation and configuration. Therefore, there is a need to provide amanner in which a user can more conveniently or easily output digitalcontent to an output device without the inconvenience of finding andinstalling new device drivers or printer drivers.

In addition, conventional output or printing methods may posesignificantly higher challenges and difficulties for mobile device usersthan for home and office users. The requirement for pre-installation ofa device-dependent driver diminishes the benefit and concept of mobile(pervasive) computing and output. For example, a mobile user may want toprint or output e-mail, PowerPoint® presentation documents, web pages,or other documents in an airplane or at an airport, gas station,convenient store, kiosk, hotel, conference room, office, home, etc. Itis highly unlikely that the user would find at any of these locations aprinter of the same make and model as is at the user's base station. Itis usually not a viable option to pre-install all of the possiblehundreds, or even thousands, of printer drivers or device driversavailable to the user's information apparatus.

As a consequence, the user would currently have to install and configurea printer driver each time at each such remote location before printing.Moreover, the user may not want to be bothered with looking for a driveror downloading it and installing it just to print out or display onepage of email at the airport. This is certainly an undesirable anddiscouraging process to promote pervasive or mobile computing.Therefore, a more convenient or automated printing and output solutionis needed so that a user can simply walk up to an output device (e.g.,printer or display device) and easily output a digital document withouthaving to install or pre-install a particular output device driver(e.g., printer driver).

Another challenge for mobile users is that many mobile informationapparatuses have limited memory space, processing capacity and power.These limitations are more apparent for small and low-cost mobiledevices including, for example, PDAs, mobile phones, screen phones,pagers, e-books, Internet Pads, Internet appliances etc. Limited memoryspace poses difficulties in installing and running large or completeprinter or device drivers, not to mention multiple drivers for a varietyof printers and output devices. Slow processing speed and limited powersupply create difficulties driving an output device. For example,processing or converting a digital document into output data by a smallmobile information apparatus may be so slow that it is not suitable forproductive output. Heavy processing may also drain or consume power orbattery resources. Therefore, a method is needed so that a small mobiledevice, with limited processing capabilities, can still reasonablyoutput digital content to various output devices.

Finally, some small mobile devices with limited display screens, such asmobile phones, may in some cases be limited to display only a few linesof text. Browsing the Internet with such devices can be a disappointingexperience when viewing, for example, complex web pages containing richformats, graphics, and images. Furthermore, some small mobile devicesmay not have appropriate applications to display complex documents orlanguages such as PDF-format files, word processing documents andPowerPoint® presentation documents etc. Typically, if an application isavailable, displaying complex original documents on small mobile devicesmay require downsizing the document or page into, for example, a fewlines of text. As an example, WAP protocol, I-Mode, and web clippingamong others may downsize, reduce or truncate information on theoriginal web page for display on mobile devices. Therefore, it isdesirable to allow mobile users to output from their small informationapparatuses to an output device the full richness of the originaldocument content.

One implementation of the present invention provides an easy, friendlyand convenient process for digital output. Unlike conventional output orprinting, a user does not have to manually pre-install a device driver(e.g., printer driver) from a CD, floppy disk, or download the driversomewhere from a network. This is well-suited for providing outputcapability to small and lower-cost mobile devices with limited memoryspace, power supply and processing capability to still be able to outputor print to an output device.

In addition, this allows small mobile devices with limited display,processing power, and memory to be able to output a digital document(e.g., PDF, HTML, PowerPoint etc) in its full original richness, withoutresorting to downsizing, truncating, reducing, clipping or otherwisealtering the original document. A user can output the original contentor document even when the small mobile device cannot display or fullydisplay the original digital document or content.

Finally, one implementation provides a convenient method allowing usersto output to an output device with or without connection to a staticnetwork. Through local communication and synchronization betweeninformation apparatus and output device, hardware and softwareinstallation for static or permanent network connectivity may not benecessary for the output device.

Additional objects and advantages of the present invention will beapparent from the detailed description of the preferred embodimentthereof, which proceeds with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a pervasive output system that canimplement the process and apparatus of the present invention.

FIGS. 2A and 2B are block diagrams illustrating exemplary configurationsof hardware components of wireless communication units.

FIGS. 3A-3C illustrate various configurations and implementations ofoutput controller with respect to an output device such as a printer.

FIG. 4 is a flow diagram of a pervasive output process of the presentinvention.

FIG. 5 is a flow diagram of an exemplary implementation of a discoveryprocess optionally included in the output process of FIG. 4.

FIG. 6 is a flow diagram of an exemplary client application processincluded in the output process of FIG. 4.

FIG. 7 is a flow diagram of an exemplary server application processutilized in the output process of FIG. 4.

FIG. 8 is a flow diagram of an exemplary final output process forpervasive output.

FIGS. 9A-9F show a series of exemplary graphical user interfaces (GUIs)rendered at different times on a display screen of an informationapparatus.

FIG. 10A is an exemplary block diagram of a conventional printing systemor printer with a printer controller.

FIG. 10B is an exemplary block diagram of a conventional output systemor an output device such as a printing system or printer that does notinclude a printer controller.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Sets forth below are definitions of terms that are used in describingimplementations of the present invention. These definitions are providedto facilitate understanding and illustration of implementations of thepresent invention and should in no way be construed as limiting thescope of the invention to a particular example, class, or category.

Objects

An object may refer to a software and data entity, which may reside indifferent hardware environments or platforms or applications. An objectmay encapsulate within itself both data and attributes describing theobject, as well as instructions for operating that data. For simplicityof discussion, an object may also include, for example, the concept ofsoftware components that may have varying granularity and can consist ofone class, a composite of classes, or an entire application.

It is important to note that the term object is not limited to softwareor data as its media. Any entity containing information, descriptions,attributes, data, instructions etc. in any computer-readable form ormedium such as hardware, software, files based on or including voice,text, graphics, image, or video information, etc., are all valid formsof object definition.

An object may also contain in one of its fields or attributes areference or pointer to another object, or a reference or pointer todata and or content. (The terms reference and pointer may be usedinterchangeably herein.) A reference to an object or any entity orcontent may include one or more, or a combination of, pointers,identifiers, names, paths, addresses or any descriptions relating to alocation where an object, data, or content can be found. Examples ofreference may include universal resource identifier scheme (URI),uniform resource locator (URL), IP address, file names, directorypointers, software object and component pointers, and run-time address,among others.

By way of example, a document object described in the present inventionmay contain or encapsulate one or more digital documents and/or one ormore pointers or references to digital documents. Therefore, moving orpassing document objects in connection with the present invention mayinclude moving or passing (1) actual digital content or (2) reference tothe actual content or (3) both. It will be appreciated that the documentobject can be quite small and lightweight if it does not also containthe digital document. These are examples of valid implementations andmay be used in the description of present invention. Differentimplementations in different situations can be easily discerned andrecognized by persons of ordinary skill in the art.

An object can reside anywhere in a network and can be transmitted todifferent environments, platforms, or applications. Downloading ortransferring an object over the network may involve protocols such asfile transfer protocol (FTP) or hypertext transfer protocol (http),among others. Transferring an object may also involve using messages orother methods through an object or component model. Three major objectsused and described in present invention are output device object (orprinter object in the case the output device is a printer), job object,and document object.

Output Device Object (or Printer Object in the case the output device isa printer)

An output device object may contain one or more attributes that mayidentify and describe, for example, the capabilities and functionalitiesof a particular output device such as a printer. An output device objectmay be stored in the memory component of an output device. As describedbelow in greater detail, an information apparatus requesting outputservice may communicate with an output device. During such local servicenegotiation, at least a partial output device object may be uploaded tothe information apparatus from the output device. By obtaining theoutput device object (or printer object in the case of a printer), theinformation apparatus may learn about the capability, compatibility,identification, and service provided by the output device.

As an example, an output device object or printer object may contain oneor more of the following fields and or attribute descriptions. Each offollowing fields may be optional, and furthermore, each of the followingfields or attributes may or may not exist in a particular implementation(e.g., may be empty or NULL).

-   -   Identification of an output device (e.g., brand, model,        registration, IP address etc.)    -   Services and feature sets provided by an output device (e.g.,        color or grayscale output, laser or inkjet, duplex, output        quality, price per page, quality of service, etc.)    -   The type of input languages, formats, and or output data (e.g.,        PostScript, PCL, XML, RTL, etc.) supported by an output device.    -   Device specific or dependent profiles (e.g., output device        profiles, printer profiles, color profiles, halftoning profiles,        communication profiles etc.). Device profiles may include        information such as color tables, resolution, halftoning, dpi        (dots-per-inch), bit depth, page size, printing speed, etc. One        or multiple profiles may exist.    -   Payment information on a plurality of services provided by an        output device.    -   Information or security requirements and type of authentication        an output device supports.    -   Date and version of the output device object (e.g., printer        object).    -   Software components containing algorithms or instructions or        data, which may be uploaded to run in an information apparatus,        as described with reference to FIG. 1. For example, a graphical        user interface (GUI) software component may be uploaded to an        information apparatus. The software component may be        incorporated into or launched in the information apparatus by a        client application of present invention to capture a user's        preferences (e.g., print quality, page layout, number of copies,        number of card per page, etc.). The client application may        capture these user preferences and create a job object, as        described below.    -   Pointer or reference to any one or more output device parameters        (output device parameters include one or more of the above        described output device object fields and or attribute        descriptions). For example, a more up-to-date or original        version of output device parameters may sometimes be stored in a        network node. An output device or may include pointer or        pointers to these output device parameters.    -   Pointer or reference to another object or objects, including        output device object, job object (described below) or document        object.

Job Object

A job object may contain attributes and information that describe anoutput job. A user may provide some or all of these attributes,preferences and or information about the output job consciously by, forexample, specifying his/her preference through a GUI or through defaultsin his/her information apparatus. Alternatively or in combination, a jobobject may be obtained without active user intervention. In oneinstance, default parameters may be provided, obtained, negotiated orcalculated without user knowledge.

Examples of attributes and information contained in a job object mayinclude one or more of the following, among others. Each of followingfields may be optional, and furthermore, each of the following fields orattributes may or may not exist in a particular implementation (e.g.,may be empty or NULL):

-   -   Preferences such as print quality, page layout, number of pages,        number of cards per page, output size, color or grayscale, among        others.    -   Information on security, authentication, payment, subscription,        identification among others.    -   Information on priority or quality of service.    -   Status information of the output job or process.    -   Job instructions such as queuing, cancellation, execution,        output priority among others.    -   Version or date of the job object.    -   Sets of default parameters or instructions. The defaults may be        stored in an output device, in the information apparatus, or in        any network nodes.    -   Pointer or reference to any one of the above mentioned        information, instructions, preferences and defaults.    -   Pointer or reference to another object or objects.

Document Object

A document object may contain attributes and fields that describe adigital document and or reference or references to digital document ordocuments. The term digital document as used herein may refer to anydigital content or data content that an output device may output. Adigital document may contain text, graphics, image, sound, voice, forms,and video, among other content types. Examples of a digital document maybe any one or combination of file types: HTML, VHTML, PostScript, PCL,XML, PDF, MS Word, PowerPoint, JPEG, MPEG, GIF, PNG, WML, VWML, CHTML,HDML, ASCII, 2-byte international coded characters, etc. A digitaldocument can be composed of any format, language, encoding, data orcombination, and the digital document may be partially or totallyproprietary or otherwise. A digital document may be used interchangeablywith the term output content or data content in the descriptions ofpresent invention.

A document object may contain one or more of the following attributes,fields, or descriptions. Each of the following fields may be optional,and furthermore, each of the following fields or attributes may or maynot exist in a particular implementation (e.g., may be empty or NULL).

-   -   The actual digital document or output content.    -   A pointer or reference to a digital document or output content        and or instructions of where a digital document can be found and        or retrieved. A digital document may be located in a user's        information apparatus, in an application, or anywhere in a        network node (e.g., in a content or file server). Using a        pointer or reference to a digital document may reduce the size        of the document object. Therefore, this may be beneficial, for        example, when passing or uploading a document object from        information apparatus to server application through a narrow        bandwidth communication link.    -   Date and version of the digital document.    -   A history with the dates and descriptions of any changes,        modifications, and updates made to the document since its        creation.    -   Descriptions and instructions for viewing, obtaining, opening,        interpreting, encoding, decoding, compressing, decompressing,        rendering, converting, describing, processing, and manipulating        the digital document.    -   Status and state of the digital document. For example, one of        the fields may provide priority or instructions of when or where        to use the reference to digital document or the actual digital        document itself if they are both contained in a document object.    -   Pointer or reference to another object or objects.

FIG. 1 is a block diagram of a pervasive output system 98 that canimplement the process and apparatus of present invention. In oneimplementation, electronic system 98 includes an information apparatus100, an application server 110 and a content server 114 (sometimesreferred to as “network nodes”) that are connected together throughnetwork 108. Electronic system 98 may also include an output device 106that communicates with information apparatus 100 through a communicationlink 116.

Network 108 generally refers to any type of wire or wireless linkbetween multiple computing devices. Examples of network 108 may include,but are not limited to, a local area network (LAN), a wide area network(WAN), or a combination of networks. In one embodiment of the presentinvention, network 108 may include the Internet. In another embodiment,network 108 may contain multiple networks, including local area networksor wide area networks such as the Internet.

Information apparatus 100 is a computing device with processingcapability. In one embodiment, information apparatus 100 may be a mobilecomputing device such as palmtop computer, handheld device, laptopcomputer, personal digital assistant (PDA), smart phone, screen phone,e-book, Internet pad, communication pad, Internet appliance, pager,digital camera, etc. It is possible that information apparatus 100 mayalso include a static computing device such as a desktop computer,workstation, server, etc.

Information apparatus 100 may contain components (not shown) such as aprocessing unit, a memory unit, a storage unit and an input/outputcontrol unit. Information apparatus 100 may also contain an interface(not shown) for interactions with users. The interface may beimplemented in software or hardware or a combination. Examples of suchinterfaces include, without limitation, one or more of a mouse, akeyboard, a touch-sensitive or non-touch-sensitive screen, push buttons,soft keys, a stylus, a speaker, a microphone, etc.

Information apparatus 100 typically contains at least one networkcommunication unit that interfaces with other electronic devices such asother nodes in network 108 or output device 106. The networkcommunication unit may be implemented with hardware (e.g., siliconchipsets, antenna), software (e.g., protocol stacks, applications) or acombination. Sometimes an information apparatus 100 may contain morethan one communication unit in order to support different interfaces,protocols, and or communication standards with different devices and ornetwork nodes. For example, information apparatus 100 illustrated inFIG. 1 may communicate with output device 106 through a Bluetoothstandard interface while communicating with other network nodes (e.g.,content server 114 or application server 110) through a cellulartelephone modem interface.

Information apparatus 100 may be coupled to network 108 through wired orwireless connections, or a combination of them. As an example,information apparatus 100 may subscribe to a wireless data network inwhich packet data is transmitted through, for example, radio linksbetween information apparatus 100 and a plurality of base stations. Awireless communication connection may include a cellular telephonecommunication channel. As another example, information apparatus 100 maybe connected to network 108 through wired lines such as, withoutlimitation, telephone lines, Ethernet, WAN links (e.g., T1, T3, 56 kb,X.25) or broadband connections (e.g., ISDN, Frame Relay and ATM), amongothers.

In one embodiment of present invention, interface 116 betweeninformation apparatus 100 and output device 106 is a wireless interface.As an example, the wireless interface may be a short-range radiointerface such as those implemented according to the Bluetooth or IEEE802.11 standard. However, the interface may be realized by other meansof wireless communication such as radio, infrared, ultrasonic orhydrophonic among others. The HomeRF Shared Wireless Access Protocol(SWAP) and one of many standards for cellular communication mayconstitute alternatives to the Bluetooth standard. Wired lineconnections such as serial or parallel interface, USB interface and firewire (IEEE 1394) interface, among others, are also possible. Connectionto a local network such as an Ethernet or a token Ring network, amongothers, may also be implemented in the present invention for localcommunication between information apparatus 100 and output device 106.Exemplary hardware components of communication units that may be used toimplement wireless interface between the information apparatus 100 andoutput device 106 are described below with reference to FIGS. 2A and 2B.

Information apparatus 100 may be a dedicated device (e.g., emailterminal, web terminal, digital camera, e-book, web pads, internetappliances etc.) with functionalities that are pre-configured bymanufacturers. Alternatively, information apparatus 100 may allow usersto install additional hardware components and or application software toexpand its functionality.

Information apparatus 100 may contain a plurality of applications toimplement its feature sets and functionalities. As an example, adocument browsing application 103 may be implemented to help a user viewand perhaps edit, partially or entirely, digital documents written incertain format or language (e.g., Page description language, markuplanguage, etc.). Digital documents may be stored locally in theinformation apparatus 100 or in a network node (e.g., in content server114). An example of a document browsing application is an Internetbrowser such as Internet Explorer, Netscape Navigator, or a WAP browser.Such browsers may use one or more standard protocols (e.g., HTTP, WAP,web clipping, I-Mode, etc.) to retrieve and display digital contentwritten in mark-up languages such as HTML, WML, XML, CHTML, HDML, amongothers. Other software applications may also constitute examples of thedocument browsing application of the present invention. For example, adocument editing software such as Microsoft Word™ also allows users toview and edit digital documents that have various file extensions (e.g.,doc, rtf, html, XML etc.) whether stored locally in the informationapparatus or in a network node.

In some instances restrictions may be imposed on the format or size ofdigital content that may be transmitted to information apparatus 100,such as when information apparatus 100 has limited processing power,screen size, memory space, or a limited application, or when bandwidthis a valuable resource in the transmission link to information apparatus100 (such as in some wireless data network). As a result, there aresituations where a user may not be able to view on information apparatus100 the full content of a digital document in its original form using adocument browsing application. For example, some images, tables,graphics, fonts and formats in a digital document may be “clipped” outor completely or partially altered from the original content before orduring the transmission process. Such restrictions may be, sometimes,imposed by a service providing the content, or by the applicationrendering the content, or by the user to avoid slow transmission etc.

To address the difficulties described above, information apparatus 100includes a pervasive output client application 102 that providespervasive output capability of the present invention. Client application102 may include software and data that can be executed by the processingunit of information apparatus 100. Client application 102 may beimplemented as a stand-alone software application or as a part orfeature of another application software, or in the form of devicedriver, which may be invoked, shared and used by other applicationsoftware. Pervasive output client application 102 may also have means toinvoke other applications (e.g., a document browsing application, acommunication manager, etc.) to provide certain feature sets, asdescribed below. Client application 102 may be variously implemented inan information apparatus 100 and may run on different operating systemsor platforms. As an example, client application 102 may include one ormore of the following functionalities:

-   -   Obtain output device object as a result of communication or        negotiation with output device 106.    -   Obtain document object (1) from user input or selection, or (2)        from other applications (e.g., a document browsing application)        residing in the information apparatus 100.    -   Coordinate with a server application 112 residing in application        server 110 to manage the process of communication and        transmission of objects or data to and from application server        112.    -   Coordinate with output device 106 that include an output        controller 104 to manage the process of transmitting output data        (or print data in the case of printers) received from the server        application 112 for output.

The client application may also optionally comprise one or more of thefollowing functionalities:

-   -   Communicate directly or indirectly (such as through an operating        system or component or object model or message, etc.) with other        applications residing in the same information apparatus 100 to        obtain objects, data, and or content needed, or relating to the        pervasive output process of present invention.    -   Directly or indirectly manage and utilize functionalities        provided by hardware components residing in its host information        apparatus such as the communication unit, storage unit, memory        unit, etc.    -   Provide a graphical user interface (GUI) in its host information        apparatus 100 to interact with user.    -   Obtain job object. Job object may be obtained (1) by user input        through a GUI, or (2) by using default values stored in a        network node or in the output device, or (3) the combination of        the above. Default values may be pre-set or may be obtained        calculated or generated by the client application as result of        communication or negotiation between client application 102,        output device 106 and or server application 112.    -   Launch or provide an interface, session or emulation for server        application 112.    -   Further process output data or print data received from server        application 112 before sending the data to output device 106 for        final output. The processing may include converting the output        data into a form (e.g., format, language, or instruction) more        acceptable to or compatible with the associated output device        106. The processing may also include at least in part one or        more raster image processing operations such as rasterization,        scaling, color management, color conversion, halftoning,        compression, decompression, etc.    -   Launch, invoke, integrate or involve a helper application to        assist at least in part in the processing of the output data.

The above functionalities and process of pervasive output clientapplication 102 are described in further detail in the pervasive outputprocess with reference to FIG. 4.

Output device 106 is an electronic system capable of outputting digitalcontent or data content regardless of whether the output medium is asubstrate (e.g., paper), display, projection, or sound. A typicalexample of output device 106 may be a printer, which outputs digitaldocuments containing text, graphics, image or any combination onto asubstrate. Output device 106 may also be a display device capable ofdisplaying still images or video, such as, without limitation,televisions, monitors, and projectors. Output device 106 can also be adevice capable of outputting sound. Any device capable of playing orreading digital content in audio (e.g., music) or data (e.g., text ordocument) formats is also a possible output device 106. A printer(including a fax machine, copier, etc.) is frequently referred to hereinas the exemplary output device 106. However, it should be recognizedthat the present invention applies also to output device 106 other thanprinters.

Outputting a data content or output content at an output device (e.g.printers, display devices, projection devices, sound output devicesetc.) includes rendering the output content on a specific output medium(e.g., papers, display screens etc). For example, rendering an outputcontent at a printer generates image on a substrate; rendering an outputcontent at a display device generates image on a screen; and renderingan output content at an audio output device generates sound.

A printing system includes three basic components: a raster imageprocessor, a memory buffer, and a marking engine. The raster imageprocessor converts digital content into a raster suitable for printing;the memory buffer holds the rasterized image ready for printing; and themarking engine transfers colorant to a substrate (e.g., paper).

Marking engine may use any of a variety of different technologies totransform a rasterized image to paper or other media or, in other words,to transfer colorant to a substrate. The different marking or printingtechnologies that may be used include both impact and non-impactprinting. Examples of impact printing may include dot matrix, teletype,daisywheel, etc. Non-impact printing technologies may include inkjet,laser, electrostatic, thermal, dye sublimation, etc.

The marking engine and memory buffer of a printer form its printerengine, which may also include additional circuitry and components, suchas firmware, software or chips or chipsets for decoding and signalconversion, etc. Input to a printer engine is usually a final rasterizedprint data that is generated by the raster image processor. Such inputis usually device dependent and printer specific. The printer engine maytake this device dependent input and generate output pages.

FIG. 10A illustrates an exemplary block diagram of one printing systemor printer 1000A that includes a printer controller 1010, a memorybuffer 1004, and a marking engine 1006. The printer controller 1010includes an interpreter 1001 and a raster image processor 1002.

The Raster image processor (RIP) may be located within an output deviceitself (as shown by raster image processor 1002 in FIG. 10A) orexternally implemented as hardware, software, or a combination (notshown). As an example, RIP may be implemented in a software applicationor device driver in the information apparatus 100. A RIP may also residewithin a printer controller 1010 (as shown by raster image processor1002 in FIG. 10A), a print server or an output controller 104 of presentinvention. Examples of raster image processing operations may includeimage and graphics interpretation, rasterization, scaling, segmentation,color space transformation, image enhancement, color correction,halftoning, compression etc.

When a RIP 1002 is located inside an output device 106, the RIP 1002 isusually included in a printer controller 1010 (as shown in FIG. 10A). Aprinter controller 1010 may interpret, convert or rasterize input printdata in the form of a page description language (e.g., PostScript, PCL),markup language (e.g., XML) or other special document format or languageinto a final format, language or instructions that printer engine 1008can understand.

A variety of other page description languages, markup languages, imageformats, graphic formats, and file formats may be used as input printdata to a printer 1000A or output device 106. Examples of possibleinputs other than PostScript and PCL may include without limitation,EMF, XML, HTML, among many others. Some printer manufacturers may alsoemploy a combination of proprietary or non-proprietary page descriptionlanguages, markup languages, file formats, graphics and image formats,color spaces, metafiles, encoding, decoding, compression ordecompression etc. for the print data. The print data sent to a printerwith printer controller 1010 is usually an intermediate description of adigital document that may require further interpretation, processing orconversion before the print data can be sent to a printer engine 1008for output. A printer controller 1010 may interpret and process theinput intermediate print data into a final format that can be understoodby the printer engine 1008. Regardless of the type of print data,conventionally, a user may need a device-specific driver in his or herinformation apparatus 100 in order to output the proper language,format, or file that can be accepted by a specific printer or outputdevice 106

FIG. 10B shows another exemplary output system or an output device1000B. An output system or output device may include one or more of aprinting system or device, a display system or device, a projectionsystem or device, or a sound system or device. In the case that theoutput system or device is a printer, the printer with reference to FIG.10A does not have a printer controller 1010. A typical example ofprinter 1000B is a lower-cost inkjet printer. RIP operations in thisexample may be implemented in a software application or device driver inan information apparatus 100 that outputs to such a printer 1000B.Conventionally, a device-specific driver or application may need toreside in the information apparatus 100 to rasterize and convert thedigital document from its original format into final print data (forexample a compressed CMKY data with one or more bits per pixel) that canbe understood by a particular printer engine 1008B.

Regardless of type or sophistication level, different output devices 106conventionally need different printer drivers or output managementapplications in the information apparatus 100 to provide outputcapability. Some mobile devices may have limited memory and processingpower to store or process multiple device drivers. It may also beinfeasible to install or preinstall multiple device dependent orspecific printer drivers in such mobile devices. In pervasive outputoperations of the present invention described below, various devicespecific drivers or applications may be available and may be executedcompletely or partially in a remote application server 110, therebyreducing the workload of information apparatus 100 and realizingdevice-independent pervasive output.

In one implementation, output device 106 includes at least acommunication unit or adapter to interface with information apparatus100, as described below in greater detail. Output device 106 maysometimes include more than one communication unit in order to supportdifferent interfaces, protocols, or communication standards withdifferent devices. For example, output device 106 may communicate with afirst information apparatus 100 through a Bluetooth interface whilecommunicating with a second information apparatus 100 through a parallelinterface, and so on. Exemplary hardware components of a wirelesscommunication unit are described below with reference to FIGS. 2A and2B.

Output device 106 may also include an output controller 104 to helpmanage communication and negotiation processes with informationapparatus 100. Output controller 104 may be dedicated hardware orsoftware or combination of both for at least one output device 106.Output controller 104 may also be integrated, installed, or connectedexternally to one or more output devices 106. In such cases the outputcontroller 104 may sometimes be referred to as print server or outputserver.

The difference between output controller 104 and printer controller 1010should be noted. Printer controller 1010 and output controller 104 areboth controllers and are both dedicated hardware and or software for atleast one output device 106. Output controller 104 refers to acontroller with feature sets, capabilities, and functionalities of thepresent invention. Printer controller 1010 may contain functions such asinterpreting an input page description language, raster imageprocessing, and queuing, among others. Output controller 104 may alsocontain partially or all the features of a printer controller 1010, plusthe feature set, functionalities, capabilities, and processes of presentinvention.

In one embodiment, output controller 104 does not include acommunication unit, but rather utilizes or manages a communication unitresiding in the associated output device 106. In another embodiment,output controller 104 may include or provide a communication unit tooutput device 106. For example, an output controller 104 with a wirelesscommunication unit may be installed internally or connected externallyto a legacy printer to provide it with wireless communication capabilitythat was previously lacking.

FIGS. 3A-3C show exemplary implementations of output controller 104 toillustrate that it may be implemented in a variety of ways. The outputcontroller 104 may be connected externally to an output device 106 orintegrated internally into the output device 106. FIG. 3A shows thatoutput controller 104 may be implemented as a circuit board or a cardthat is installed inside an output device 106 and may include software,hardware, or both. FIG. 3C shows that output controller 104 may beimplemented as an external box or station that is wired or wirelesslyconnected to an output device 106. Such an external box or station maycontain its own user interface. One example of such an implementation isa print server connected to an output device 106. FIG. 3B shows anotherconfiguration in which the functionalities of output controller 104 maybe integrated into an existing printer controller 1010 (referred to as“combined controller”) which can be internally or externally (not shown)connected to output device 106. A combined controller 104B hasfunctionalities of both printer controller 1010 (e.g., inputinterpretation and or raster image processing) and output controller 104of the present invention. Under this configuration, the functionalitiesof output controller 104 and printer controller 1010 may share the sameresources, such as processing unit, memory unit, etc.

Other possible implementations of output controller 104 may include, forexample, a conventional personal computer (PC), a workstation, and anoutput server or print server. In these cases, the functionalities ofoutput controller 104 may be implemented using application softwareinstalled in a computer (e.g., PC, server, or workstation), with thecomputer connected with a wired or wireless connection to an outputdevice 106. Using a PC, server, workstation, or other computer toimplement the feature sets of output controller 104 with applicationsoftware is just another possible embodiment of the output controller104 and in no way departs from the spirit, scope and process of thepresent invention.

Regardless of its manner of implementation, the output controller 104will usually include hardware, software, or a combination. For example,an output controller 104 may include components using one or more orcombinations of an application-specific integrated circuit (ASIC), adigital signal processor (DSP), a field programmable gate array (FPGA),firmware, system on a chip, and various communication chip sets. Outputcontroller 104 may also contain embedded processors with softwarecomponents or embedded application software to implement its featuresets, and functionalities.

Output controller 104 may contain an embedded operating system. With anoperating system, some or all functionalities and feature sets of theoutput controller 104 may be provided by application software managed bythe operating system. Additional application software may be installedor upgraded to newer versions in order to, for example, provideadditional functionalities or bug fixes.

Output controller 104 typically includes a memory unit, or may share amemory unit with, for example, printer controller 1010. The memory unit,such as ROM, RAM, flash memory and disk drive among others, may providepersistent or volatile storage. The memory unit may store objects,codes, instructions or data (collectively referred to as softwarecomponents) that implement the functionalities of the output controller104. Part of the software components (e.g., the printer object) may beuploaded to information apparatus 100 during a data output operation.

Functionalities and components of output controller 104 for the purposeof providing pervasive output may include:

-   -   Input components and operations for receiving service requests        from a plurality of information apparatuses 100.    -   Storage components and operations for storing partial or the        entire printer or output object in a memory component.    -   Transmission components and operations for transmitting partial        or the entire printer or output object to the information        apparatus 100 requesting pervasive output service. The output        controller 104 may transmit the output device object in one or        multiple sessions.    -   Receiving components and operations for receiving output data        (e.g., print data) from information apparatus 100 and sending        the data to output device 106 or output engine (or printer        engine) 1008.

In addition to the above functionalities, output controller 104 mayfurther optionally include one or more of the following:

-   -   Response components and operations to respond to service request        from an information apparatus 100 (e.g., in a discovery process,        described below) by providing at least a partial output device        object (e.g., printer object for printers).    -   Broadcast components and operations to broadcast or advertise        the services provided by a host output device 106 to an        information apparatus 100 that may request such services.    -   Payment components and operations for implementing payment        processing and management functions by, for example, calculating        and processing payments according to the services requested or        rendered to a client (information apparatus 100).    -   Components and operations for implementing job management        functionalities such as queuing and spooling among others.    -   User interface components and operations for providing a user        interface (e.g., display screen, touch button, soft key, etc.)        when it is implemented as an external box connected to an output        device 106.    -   Security components and operations for implementing security or        authentication procedures. For example, the output controller        104 may store in its memory component (or shared memory        component) an access control list, which specifies what device        or user may obtain service from its host (or connected) output        device 106. Therefore, an authorized information apparatus 100        may gain access after confirming with the control list.    -   Processing components and operations to further process the        output data. The processing of the output data may include        converting the output data into a form (e.g., format, language,        and or instruction) at least more acceptable to or compatible        with the associated output device 106 or output engine or        printer engine 1008. The processing may also include at least in        part one or more raster image processing operations such as        rasterization, scaling, color management, color conversion,        halftoning, compression, decompression etc.

When output controller 104 is implemented as firmware, or an embeddedapplication, the configuration and management of the functionalities ofoutput controller 104 may be optionally accomplished by, for example,using controller management software in a host computer. A host computermay be a desktop personal computer (PC), workstation, or server. Thehost computer may be connected locally or through a network to theoutput device 106 or the controller 104. Communication between the hostcomputer and the output controller 104 can be accomplished through wiredor wireless communication. The management application software in thehost computer can manage the settings, configurations, and feature setsof the output controller 104. Furthermore, host computer's configurationapplication may download and or installed application software, softwarecomponents and or data to the output controller 104 for the purpose ofupgrading, updating, and or modifying the features and capabilities ofthe output controller 104.

Output device 106 in one implementation includes or is connected tooutput controller 106 described above. Therefore, functionalities andfeature sets provided by output controller 106 are automaticallyincluded in the functionalities of output device 106. The output device106 may, however, implement or include other controllers and/orapplications that provide at least partially the features andfunctionalities of the output controller 104.

Therefore, the output device 106 may include some or all of thefollowing functionalities:

-   -   Components and operations to receive multiple service requests        or queries (e.g., a service request, a data query, an object        query etc.) from a plurality of information apparatus 100 and        properly respond to them by returning software components, which        may contain data, codes, instructions and/or objects.    -   Components and operations to receive, from information apparatus        100 device-specific output data containing the digital document        intended for output.    -   Components and operations to output final output data or print        data on a substrate or in another medium such as display screen.

An output device may further comprise optionally one or more of thefollowing functionalities:

-   -   Components and operations for establishing and managing a        communication link with an information apparatus 100 requesting        service;    -   Components and operations to store and/or further process input        print data or output data.    -   Components and operations for storing partial or the entire        output device object (e.g. printer object) in a memory        component.    -   Components and operations to advertise or broadcast services        provided or available to an information apparatus 100 that may        request such services.    -   Components and operations for implementing payment processing        and management functions by, for example, calculating and        processing payments according to the services requested by or        rendered to a client (information apparatus 100).    -   Components and operations for implementing job management        functionalities such as queuing and spooling among others.    -   Components and operations for providing a user interface (e.g.,        display screen, touch button, soft key, power switch, etc.).    -   Components and operations for implementing security or        authentication procedures. For example, the output device 106        may store in its memory component (or a shared memory component)        an access control list, which specifies what device or user may        obtain service from it. Therefore, an authorized information        apparatus 100 may gain access after confirming with the control        list.    -   Components and operations to process the output data. The        processing of the output data may include converting the output        data into a form (e.g., format, language, and or instruction)        more acceptable or compatible to the associated printer        controller 1010, output engine or printer engine 1008. The        processing may also include at least partially one or more        raster image processing operations.

Application server 110 is a node on network 108. An application server110 may include computing capability, data storage capability, andmechanisms for servicing requests from a plurality of client computers(referred to as clients), including the information apparatus 100,needing computational or data storage resources. A server typicallyincludes processing unit, memory unit, storage unit, input/outputcontrol unit, and a communication unit, among others. Application server110 may also include an interface to interact with users. The interfacemay be implemented with, for example, display screen, touch-sensitivescreen, keyboard, mouse, stylus, push button, microphones and speakersamong others.

Application server 110 preferably includes at least an operating systemfor supporting a plurality of application software to implement thefunctionalities of the application server 110 and to provide services toits clients (e.g., information apparatus 100). To provide services tomultiple clients at the same time, the application server 110 may run amulti-user operating system that enables multiple concurrent users tolog on and run applications in separate, protected sessions.

Application server 110 typically includes a server application 112 thatmay include software and data to be executed in the processing unit of aserver node. The server application 112 may include one or more of thefollowing functionalities:

-   -   Components and operations to receive data and/or objects (with        at least a output device object and a document object) from        client application 102.    -   Components and operations to process the objects received to        generate device-dependent output data acceptable to one or more        output devices 106 selected by a user. The server application        112 may perform the processing function independently or in        combination with other applications.    -   Components and operations to transmit output data back to the        client application 102.

The server application 112 may optionally include one or more of thefollowing functionalities:

-   -   Components and operations to obtain digital document (output        content) from other network node if it is not included or        completely included in the document object received from client        application 102.    -   Components and operations to obtain output device parameters if        they are not included or completely included in the output        device object received from client application 102.    -   Components and operations to obtain job object or relating        parameters if it is not included or completely include in the        composite message received from client application 102. The        server application 112 may obtain job object by querying user or        by assuming default values. For example, if a user did not        specify print or output range, the server application 112 may        assume the default by printing or outputting all pages.    -   Components and operations to manage and coordinate communication        with an information apparatus 100 requesting output service.    -   Components and operations to identify, invoke and or incorporate        appropriate applications (residing in the application server 112        or in other network nodes) to help interpret and or process the        digital document obtained and generate output data. The        processing of the digital document may include converting the        digital document into an output data related at least in part to        the output device object. The processing may also include at        least in part one or more raster image processing operations        such as rasterization, scaling, color management, color        conversion, halftoning, compression, decompression, etc.    -   Components and operations to interact with (e.g., store, update,        verify, etc.) one or more databases that store profiles of        information apparatuses 100 and/or users who have subscribed to        output service provided by the server application 112.    -   Components and operations to implement an independent computing        architecture where the execution of the server application 112        may occur entirely in the application server 110. The GUI of the        application 112, keystrokes and mouse clicks are transmitted        over the network 108 to and from the client such as a user's        information apparatus 100.

It should be noted that, in the example shown here, server application112 is illustrated as the only application in application server 110.But in actuality, more than one application may exist in the applicationserver 110 and the applications may provide various services todifferent clients. It is also possible that the functionalities andfeature sets of the server application 112 may be implemented withmultiple applications residing in the same server, multiple servers(e.g., in a server farm), or network nodes or combination.

Content server 114 may represent one of a plurality of server nodes onnetwork 108 that may store digital documents 116. The digital documentsstored in content server 114 may be viewed or edited by a user using aninformation apparatus 100. As an example, the content server 114 may bea web server that hosts a plurality of web pages written in mark uplanguages such as HTML, WML, XML, HDML, CHTML, among others. A user mayview web pages using an Internet browsing application such as InternetExplorer or Netscape Navigator, a WAP browser, etc. As another example,the content server 114 may be a file server that allows multiple clientsto store and share digital files with appropriate security orauthentication procedures. These digital files or documents may containone or more of image, text, graphics, sound and video. The files may besaved in various file formats (e.g., MS Word, Excel, PowerPoint, PDF,Postscript, JPEG, GIF, MPEG, etc.). A user may need to have appropriateapplication on his/her information apparatus 100 to access, view andedit these files.

It should be noted that in FIG. 1 content server 114 and applicationserver 110 are shown as distinct server nodes. However, it is possiblethat application server 110 and content server 114 belong to the samedomain, or the functionalities of these two servers may be implementedwith different applications executed in a single network node or server.

Further description of the functionalities and feature sets of differentdevices and applications illustrated in FIG. 1 is provided below inreference to a pervasive output process shown in FIG. 4.

FIGS. 2A and 2B are block diagrams illustrating two exemplaryconfigurations of hardware components of wireless communication units.Referring to FIG. 2A, a radio adapter 200 may be implemented to enabledata/voice transmission among devices (e.g., information apparatus 100and output device 106) through radio links. A RF transceiver 214 coupledwith antenna 216 is used to receive and transmit radio frequencysignals. The RF transceiver 214 also converts radio signals into andfrom electronic signals. The RF transceiver 214 is connected to a RFlink controller 210 by an interface 212. The interface 212 may performfunctions such as analog-to-digital conversion, digital-to-analogconversion, modulation, demodulation, compression, decompression,encoding, decoding, and other data or format conversation functions.

RF link controller 210 implements real-time lower layer (e.g., physicallayer) protocol processing that enables the hosts (e.g., informationapparatus 100, output controller 104, output device 106, etc.) tocommunicate over a radio link. Functions performed by the linkcontroller 210 may include, without limitation, errordetection/correction, power control, data packet processing, dataencryption/decryption and other data processing functions.

A variety of radio links may be utilized. A group of competingtechnologies operating in the 2.4 GHz unlicensed frequency band is ofparticular interest. This group currently includes Bluetooth, Home radiofrequency (Home RF) and implementations based on IEEE 802.11 standard.Each of these technologies has a different set of protocols and they allprovide solutions for wireless local area networks (LANs). Interferenceamong these technologies could limit deployment of these protocolssimultaneously. It is anticipated that new local area wirelesstechnologies may emerge or that the existing ones may converge.Nevertheless, all theses existing and future wireless technologies maybe implemented in the present invention without limitation, andtherefore, in no way depart from the scope of present invention.

Among the current available wireless technologies, Bluetooth may requirerelatively lower power consumption. Bluetooth has its own protocol stackand is designed for short range (10 meters), point-to-multipoint voiceand data transfer. It is based on a frequency-hopping version of spreadspectrum. Seventy-nine hop frequencies are utilized beginning at thelowest frequency of 2402 MHz and each of the 79 hop frequencies is 1 MHzabove the next lower frequency. Bluetooth-enabled devices operate inpiconets, in which several devices, using the same hopping pattern orsequence, are connected in a point-to-multipoint system (piconet). Thereis one device (master) in each piconet that determines how the bandwidthis allocated to other devices (slaves). As many as 10 piconets of 8devices each can operate simultaneously.

Referring to FIG. 2B, one or more infrared (IR) adapters 220 may beimplemented to enable data transmission among devices through infraredtransmission. The IR adapters 220 may be conveniently implemented inaccordance with the Infrared Data Association (IrDA) standards andspecifications. In general, the IrDA standard is used to providewireless connectivity technologies for devices that would normally usecables for connection. The IrDA standard is a point-to-point (vs.point-to-multipoint as in Bluetooth), narrow angle, ad-hoc datatransmission standard designed to operate over a distance of 0 to 1meter and at speeds up to 4 Mbps.

Configuration of infrared adapters 220 may vary depending on theintended rate of data transfer. FIG. 2B illustrates one embodiment ofinfrared adapter 220. Transceiver 226 receives/emits IR signals andconverts IR signals to/from electrical signals. A UART (universalasynchronous receiver/transmitter) 222 performs the function ofserialization/deserialization, converting serial data stream to/fromdata bytes. The UART 222 is connected to the IR transceiver 226 byencoder/decoder (ENDEC) 224. This configuration is generally suitablefor transferring data at relatively low rate, for example 115.2 kbps orbelow. Other components (e.g., packet framer, phase-locked loop) may beneeded for higher data transfer rates.

FIGS. 2A and 2B illustrate exemplary hardware configurations of wirelesscommunication units. Such hardware components may be included in devices(e.g., information apparatus 100, output controller 104, output device106, etc.) to support various wireless communications standards. Wiredlinks, however, such as parallel interface, USB, Firewire interface,Ethernet and token ring networks may also be implemented in the presentinvention by using appropriate adapters and configurations.

FIGS. 3A-3C illustrate various configurations and implementations ofoutput controller 104.

In the configuration illustrated by FIG. 3A, output controller 104A maybe installed, for example as one or more boards or cards containing bothhardware and software, inside output device 106(A). The output device106(A) may or may not have inside it a printer controller 1010 (FIG.10). In the case that output device 106(A) includes a printer controller1010 (not shown), the output controller 104 may co-exist with theprinter controller 1010 and other components of the output device106(A). One example of this implementation is to connect or cascadeoutput controller 104A sequentially or serially with the printercontroller 1010.

In another implementation shown by FIG. 3B, the functionalities ofoutput controller 104 and printer controller 1010 (FIG. 10) may becombined into a single controller, sometimes referred to as combinedcontroller 104(B). The combined controller 104(B) may be implemented,for example, as one or more boards or cards containing both hardware andsoftware, inside output device 106(B). This implementation may help toreduce the cost of material when compared to implementing two separatecontrollers. As an example, the combined controller 104(B) may sharecommon processors, memories, and storage units to run the applicationsand functionalities of the two types of controllers and, therefore, mayhave lower component cost. The combined controller can also beimplemented as an external box or station (not showed) connected to theoutput device 106. This connection could be wired or wireless.

In the third implementation shown in FIG. 3C, the output controller 104Cmay be implemented in a separate box or server or station connectedexternally to output device 106C. T he communication link between outputcontroller 104C and output device 106C may be a wired or wireless link.The output device 106C may or may not have inside it a printercontroller 1010 (FIG. 10). If the output device 106C does not contain aprinter controller 1010 inside, the externally connected outputcontroller 104C may optionally contain the functionalities of printercontroller 1010 and therefore, provide raster image processingcapability as an additional feature.

The above are exemplary implementations and configurations of outputcontroller 104. Other implementations are also possible. For example,partial functionalities of output controller 104 may be implemented inan external box or station while the remaining functionalities mayreside inside an output device 106 as a separate board or integratedwith a printer controller 1010. As another example, the functionalitiesof output controller 104 may be implemented into a plurality of externalboxes or stations connected to the same output device 106. As a furtherexample, the same output controller 104 may be connected to service aplurality of output devices 106. Variously implemented outputcontrollers 104 should not depart from the spirit and scope of thepresent invention provided that they all support the functionalities andfeature sets described herein.

FIG. 4 is a flow diagram of a pervasive output process 401 of thepresent invention. Pervasive output process 401 allows an informationapparatus 100 to output digital content or document in its original formto an output device 106 regardless of processing power, display screensize, or memory space of information apparatus 100. Pervasive outputprocess 401 may include or utilize:

-   -   A client application 102, in an information apparatus 100,        obtaining objects; with at least one object including a document        object and another object including an output device object (or        printer object in the case of a printer);    -   A client application 102 transmitting objects to a server        application 112;    -   A server application 112 obtaining and processing the document        object and converting it into output data, reflecting at least        in part a relationship to said output device object;    -   A server application 112 transmitting output data to the        information apparatus 100 including a client application 102;    -   The information apparatus 100 including a client application 102        transmitting output data to an output device 106;    -   An output device 106 generating output with the output data.

Pervasive output process 401 may be initiated by a user in step 400.Typically, a user initiates output process 401 by invoking a clientapplication 102 in his/her information apparatus 100. The clientapplication 102 may be launched as an independent application or it maybe launched from other applications 103 (such as from a documentbrowsing, creating or editing application) or as part of or component ofor a feature of another application 103 residing in the same informationapparatus 100. When launched from another application 103, the clientapplication 102 may obtain information, such as a partial or an entiredocument object, from that another application 103. This can beaccomplished, for example, by one or combinations of messages orfacilitated through an operating system or a particular object orcomponent model etc. The client application maybe a device driver inthis example.

During output process 401, a user may need to select one or more outputdevices 106 for output service. An optional discovery process step 404may be implemented to help the user select an output device 106. Duringthe discovery process step 404, a user's information apparatus 100 may(1) search for available output devices 106; (2) provide the user with alist of available output devices 106; and (3) provide means for the userto choose one or more output devices 106 to take the output job. Anexemplary discovery process 404 is described below in greater detailwith reference to FIG. 5.

The discovery process 404 may sometimes be unnecessary. For example, auser may skip the discovery process 404 if he or she already knows theoutput device (e.g., printer) 106 to which the output is to be directed.In this case, the user may simply connect the information apparatus 100to that output device 106 by wired connections or directly point to thatoutput device 106 in a close proximity such as in the case of infraredconnectivity. As another example, a user may pre-select or set theoutput device or devices 106 that are used frequently as preferreddefaults. As a result, the discovery process 404 may be partially orcompletely skipped if the default output device 106 or printer is foundto be available.

In stage 406, the client application 102 may interact with output device106, the user, and or other applications residing in the sameinformation apparatus 100, to obtain (1) a document object, (2) anoutput device object and (3) any other optional objects such as a jobobject. These objects may be obtained sequentially, concurrently, or inany order. It should be noted that some of the objects or partialobjects might have been acquired in prior steps. For example, the clientapplication 102 may have obtained partially or entirely document objectsor related information when the client application 102 was launched fromor by another application. As another example, a partial output deviceobject may have been uploaded to the information apparatus 100 duringthe optional discovery process 404. The client application 102 maycreate a composite message including these objects (document object,output device object and other optional objects) and transmit thecomposite message to server application 112 for processing, as describedbelow in greater detail with reference to FIG. 6.

The server application 112, after receiving such a composite messagefrom the client application 102, may in step 408 processes the documentobject or objects contained in the composite message and convert it orthem into output data. Additional helper applications may be needed tohelp in this processing task. The processing and generation of outputdata may reflect at least in part a relationship to the output deviceobject and or job object contained in the composite message receivedfrom client application 102. The output data generated may betransmitted back to the information apparatus 100, requesting outputservice or process 401 via network 108. An exemplary implementation of aserver application process is described below with reference to FIG. 7.

In step 410, information apparatus 100 transmits output data, with orwithout further processing, to the selected output device 106 through alocal communication link 116. Preferably, the output data is in a formator language acceptable to or compatible with the output device 106selected by the user. When receiving output data from informationapparatus 100, an output device 106 may simply buffer the output databefore sending it to output engine (or printer engine) 1008 for finaloutput. In cases where an output device 106 includes or is connected toa printer controller 1010, output controller 104 or a combinedcontroller, such controllers may further process the output datareceived from information apparatus 100 before sending the output datato printer engine 1008 for final output. Further processing, if itexists, may include at least partially conversion operations and orraster image processing operations on the output data. The outputcontroller 104 may also provide queuing or spooling or other jobmanagement features. An exemplary implementation of such an outputprocess is described below with reference to FIG. 8.

The steps included in pervasive output process 401 may proceedautomatically when a user requests output service. Alternatively, a usermay be provided with options to proceed, cancel, or input information ateach and every step. For example, after the client application processstep 406 is finished, the user may be asked whether or not he or shewishes to continue to communicate with a remote application server. Asanother example, a user may cancel the output service at any time by,for example, indicating a cancellation signal or command or byterminating the client application or by shutting down the informationapparatus 100 etc.

A printer is used as a primary output device 106 in this example. Itshould be recognized, however, that pervasive output process 401described above can be similarly applied to other output devices such asfax machines, digital copiers, display screens, monitors, televisions,projectors, voice output devices, among others.

FIG. 5 is a flow diagram of an exemplary implementation of a discoveryprocess 520, which may be an optional step to help a user locate one ormore output devices 106 for an output job. The discovery process 520may, however, be skipped partially or entirely. Implementation ofdiscovery process 520 may require compatible hardware and softwarecomponents residing in both the information apparatus 100 and the outputdevice 106.

The discovery process 520 may include or utilize:

-   -   A client application 102 in an information apparatus 100        communicating with available output devices 106 to obtain        information such as partial or complete output device object or        objects.    -   A client application 102 that provides to the user information        on each available and or compatible output device 106.    -   User selection or determination by a client application 102        (automatically or not) of one or more devices 106 for output        service from the available or compatible output devices 106.

Various protocols and or standards may be used during discovery process520. Wireless communication protocols are preferred. Wiredcommunication, on the other hand, may also be implemented. Examples ofapplicable protocols or standards may include, without limitation,Bluetooth, HAVi, Jini, Salutation, Service Location Protocol, andUniversal Plug-and-play among others. Other proprietary protocols orcombination may also be implemented in the discovery process 520.However, these different protocols, standards, or combination shall notdepart from the spirit and scope of present invention.

In one implementation an application (referred here for simplicity ofdiscussion as a “communication manager,” not shown) residing in theinformation apparatus 100 helps communicate with output device 106,manage service requests and discovery process 520. The communicationmanager may be a part of or a feature of the client application 102.Alternatively or in combination, the communication manager may also be aseparate application. When the communication manager is a separateapplication, the client application 102 may have the ability tocommunicate, manage or access functionalities of the communicationmanager.

The discovery process 520 may be initiated manually by a user orautomatically by a communication manager when the user requests anoutput service with information apparatus 100.

In the optional step 500, a user may specify searching or matchingcriteria. For example, a user may indicate to search for color printersand or printers that provide free service. The user may manually specifysuch criteria each time for the discovery process 520. Alternatively orin combination, a user may set default preferences that can be appliedto a plurality of discovery processes 520. Sometimes, however, nosearching criteria are required—the information apparatus 100 may simplysearch for all available output devices 106 that can provide outputservice.

In step 501, information apparatus 100 searches for available outputdevices 106. The searching process may be implemented by, for example,an information apparatus 100 multi-casting or broadcasting oradvertising its service requests and waiting for available outputdevices 106 to respond. Alternatively or in combination, an informationapparatus 100 may “listen to” service broadcasts from one or more outputdevices 106 and then identify the one or more output devices 106 thatare needed or acceptable. It is also possible that multiple outputdevices 106 of the same network (e.g., LAN) register their services witha control point (not shown). A control point is a computing system(e.g., a server) that maintains records on all service devices withinthe same network. An information apparatus 100 may contact the controlpoint and search or query for the needed services.

In step 502, if no available output device 106 is found, thecommunication manager may provide the user with alternatives 504. Suchalternatives may include, for example, aborting the discovery process520, trying discovery process 520 again, temporarily halting thediscovery process 520, or being notified when available output device106 is found. As an example, the discovery process 520 may not detectany available output device 106 in the current wired/wireless network.The specified searching criteria (if any) is then saved or registered inthe communication manager. When the user enters a new network havingavailable output devices 106, or when new compatible output devices 106are added to the current network, or when an output device 106 becomesavailable for any reason, the communication manager may notify the userof such availability.

In step 506, if available output devices 106 are discovered, thecommunication manager may obtain some basic information, or part of orthe entire output device object, from each discovered output device 106.Examples of such information may include, but not limited to, deviceidentity, service charge, subscription, service feature, devicecapability, operating instructions, etc. Such information is preferablyprovided to the user through the user interface (e.g., display screen,speaker, etc.) of the information apparatus 100.

In step 508, the user may select one or more output devices 106 to takethe output job. If the user is not satisfied with any of the availableoutput device 106, the user may decline the service. In this case, theuser may choose to try again in step 510 with some changes made to thesearching criteria. Alternatively, the user may choose to terminate theservice request.

In step 512, with one or more output devices 106 selected or determined,the communication link between information apparatus 100 and theselected output device or devices 106 may be “locked”. Other outputdevices 106 that are not selected may be dropped. The output process 520may then proceed to a client application process 601 referenced in step406 of FIG. 4.

FIG. 6 is a flow diagram of an exemplary client application process 601that may include or utilize:

-   -   A client application 102 that obtains an output object or        objects as a result of communication and or negotiation with an        output device 106.    -   A client application 102 that obtains a document object (1) from        user input or selection, or (2) from other applications (e.g.,        document browsing application) residing in the same information        apparatus 100.    -   A client application 102 that coordinates with server        application 112 to manage the process of communication and        transmission of a composite message (including at least a        printer object and a document object) to the server application        112 for further processing.

Step 600 indicates that client application 102 obtains a documentobject. To assemble or create a document object, the client application102 may need to obtain output content (or a digital document) orpointers to the output content, or both. In cases where the outputcontent is downloaded or partially downloaded to a user's informationapparatus 100, the client application 102 may include both the outputcontent and pointers to the output content in its document object. Incases where the output content is not downloaded to the informationapparatus 100, the client application 102 may include only pointers tothe output content in the document object.

There are various ways that a client application 102 may obtain adigital document or a pointer or reference to the digital document. Inone embodiment, the client application 102 may provide a GUI with whicha user can directly input the pointer or reference (e.g., URL, IPaddress, filename, path, etc.) of a digital document stored locally orin a network node. This manual process may be facilitated by, forexample, providing a GUI with which users may select one or morepointers from a list of pointers or references of digital documentsstored locally or in a remote network node. For instance, through a GUIprovided by the client application 102, a user may see and selectpointers of digital documents stored in a remote file server. In thiscase, the document object created by the client application 102 maycontain only pointers or references to the output content.

In another embodiment, the client application 102 may obtain outputcontent or pointer to output content from another application in thesame information apparatus 100. As an example, a user may (1) launch theclient application 102, and (2) invoke another application 103 (e.g.,document editing and or browsing application) residing in the sameinformation apparatus 100 to view or download the digital document. Asanother example, a user may (1) run another application 103 (e.g.,document editing and or browsing application) residing in the sameinformation apparatus 100 to view or download the digital document; and(2) launch or invoke the client application 102. In these cases, theclient application 102 may communicate with another application 103(e.g., document browsing application) to obtain pointers to the digitaldocument and or the digital document itself (if it has been downloadedlocally for viewing) to be included in document object.

It should be noted that the document object may have been partially orentirely obtained by the client application 102 in previous steps suchas in step 400 of FIG. 4 in which the client application 102 isinitiated. As an example, a user may (1) view or download a digitaldocument (stored locally or in a network node) by using a documentbrowsing application on the information apparatus 100, and (2) requestoutput service by launching the client application 102 (as in step 400).The client application 102 may then communicate with the documentbrowsing application to obtain the document object (including digitaldocument and or pointers to the digital document). In this case, step600 may be partially or entirely skipped.

In some instance such as in document browsing applications, a user maybe limited to viewing or downloading only part of, or a reduced versionof, the original digital document stored in a network node. This may bedue to small screen size, limited bandwidth, memory size, and lack ofapplication support, among other issues. In these cases, the clientapplication 102 may allow the user to choose to output the originaldigital document or the reduced sized document. In the case of theoriginal document, the document object created by the client application102 may, for example, contain only references or pointers to theoriginal output content stored in a server or network node. In the casewhere user wants to output the reduced version document that has beendownloaded locally, the client application 102 may include in thedocument object one or more of (1) the obtained reduced output content(2) the reference to the reduced content in the server or network node,and (3) pointer or pointers to the original output content.

In another instance, for example, with a document browsing application,a user may see only the name or path to output content without beingable to download or open it due to, for example, no compatibleapplications residing on the information apparatus 100, small displayscreen etc. According to the present invention, however, a user maystill be able to output the original digital document by indicating tothe client application 102 the pointer or reference of the outputcontent. For example, the user may select or highlight the name or pathof the output content through a GUI provided by the document browsingapplication. In this case, the document object created by the clientapplication 102 may contain only pointers to the output content.

A document object may also contain instructions. Instructions in anobject may provide description, operation, and status information of thecontent or data of the document object, etc. For example, instructionsmay provide information about the changes or differences between anoutput content included in a document object relative to the originaloutput content (stored in a network node) that is pointed to by thereference field in the document object. The output content included in adocument object may be an edited or altered version while the outputcontent referred to by the pointers may be the original version.

It should be noted that some output content may be publicly available toall users (e.g., generic web pages) while other output content may berestricted to one or a group of users (e.g., secure documents stored ina corporate network). In the latter case, authentication informationsuch as a password, user name, id number, biometric information, digitalcertificate or security key, among others, may need to be provided tothe server application 112 for accessing and or fetching the digitaldocument with reference to a pointer or reference. In one example, suchauthentication information may be included in document object, printerobject, or job object. In another example, server application 112 mayprompt the user to enter authentication information through a GUI ininformation apparatus 100 when necessary.

Step 602 indicates whether an output device object is obtained. In thecase where an output device object is not obtained, then in step 604,with one or more output devices 106 selected, the client application 102may communicate with the selected output device or devices 106 to uploadoutput device objects or related information stored in memory or storagecomponents of the output device or devices 106. It should be noted thata partial or entire output device object or related information mighthave been already obtained by the client application 102 during theprior optional discovery process (520 and step 402 in FIG. 4). In thiscase, step 604 may be partially or entirely skipped.

To successfully obtain the output device object or objects from theselected output device or devices 106, several additional optionalprocesses may be involved. As an example, authentication may benecessary when the selected output device 106 provides service to arestricted group of users. A simple authentication may be implementedby, for example, comparing the identity for the information apparatus100 with an approved control list of identities or elements stored inthe output device 106. Other more complex authentication and encryptionschemes may also be used. Information such as user name, password, IDnumber, signatures, security keys (physical or digital), biometrics,fingerprints, voice, among others, may be used separately or incombination as authentication means. Such identification and orauthentication information may be manually provided by the user orautomatically detected by the selected output device or devices 106.With successful authentication, a user may gain access to all or part ofthe services provided by an output device 106. The output device objectthat the client application 102 obtains may vary according to the typeor quality of service requested or determined. If authentication fails,it is possible that a user may be refused partially or completely allaccess to the service. In this case, the user may be provided withalternatives such as selecting another output device 106 or alternativeservices.

Another optional process is that a user may be asked to provide paymentor deposit or escrow before, during or after output service. Examples ofpayment or deposit may include cash, credit card, bankcard, charge card,smart card, electronic cash, among others. The output controller 104 mayprovide payment calculation or transaction processing as optionalfeature sets.

It should be noted that a plurality of information apparatuses 100 mayrequest to obtain output device object or objects from the same outputdevice 106 at the same time or at least during overlapping periods. Theoutput device 106 may have components or systems to manage multiplecommunication links and provide the output device object or objectsconcurrently or in an alternating manner to multiple informationapparatuses 100. Alternatively, an output device 106 may providecomponents or systems to queue the requests from different informationapparatuses 100 and serve them in a sequential fashion according to ascheme such as first come first serve, quality of service, etc.Multi-user communication and service management capability with orwithout queuing or spooling functions may be implemented by, forexample, the output controller 104 as optional feature sets.

Step 606 indicates that the client application 102 may optionally obtaina job object. A job object may include a user's preferences andparameters relating to the output job or process. The client application102 may obtain a job object by, for example, capturing a user's outputpreferences through a GUI. In one instance, the client application 102may provide a universal GUI to a user regardless of what output device106 is selected. Through such an interface, the user may specify somedevice-independent parameters such as page range, number of cards perpage, number of copies, etc. Alternatively or in combination, the clientapplication 102 may incorporate device-dependent features andpreferences into the GUI provided to user. The device dependent portionof the GUI may be supported partially or entirely by some informationcontained in or provided by the output device object obtained from theselected output device 106. Examples of such device dependent featuresmay include quality of service, service fee, print quality, color orgrayscale, duplex or single sided, output page size, among others.

It is possible that some or all components, attributes or fields of ajob object have default values. The client application 102 may havecertain defaults such as those hard-coded in software or hardware orpre-configured by the user or a manufacturer. In addition, the clientapplication 102 may access a file to obtain default values or by othermeans such as communicating with the output device 106, the serverapplication 112 or other applications in the information apparatus 100or in a network node. The client application 102 may also create orcalculate certain default values based on the information it has orobtained during the output process. These default values may be related,at least in part, to the output device object and or the serverapplication 112 involved in the output process. In some instances, auser may or may not have an opportunity to change or overwrite some orall defaults. The client application 102 may obtain and use some or alldefaults with or without user intervention or knowledge.

In step 608, the client application 102 may create or assemble acomposite message. A composite message may be any type of datatransferred across network 108 that may include one or moretransmissions. A composite message typically includes partially orentirely the objects (with some default values) obtained by the clientapplication 102 in previous steps.

In step 610, the client application 102 transmits the composite messageto server application 112. The client application 102 may communicatewith the server application 112 using one or more or a combination ofstandard network protocols such as WAP, Web Clipping, I-Mode, TCP/IP,SPX/IPX, PPP, NetBEUI, Apple Talk, among others. Proprietary networkprotocols or a combination that includes them may also be used. Thecommunication link between information apparatus 100 and applicationserver 112 may be implemented with one or a combination of standardnetwork connections and communication links such as telephone lines, LANor WAN links (T1, T3, 56 kb, X.25, etc.), broadband connections (ISDN,Frame Relay, ATM etc), wireless connection (radio link, inferred,microwave, etc.) as well as the Internet or corporate Intranets.

In addition to the composite message, the client application 102 andserver application 112 may also exchange various types of messages backand forth to, for example, request service, confirm serviceavailability, configure protocol stack, or confirm or acknowledgereceipt of the previous message, among other messages. The serverapplication 112 may also prompt the client application 102 to send orresend some components or objects of the composite message if suchcomponents or objects are incomplete, missing or corrupted. The serverapplication 112, however, may also try to fill in default values (ifavailable) for some or all of the information missing in the compositemessage. Encode/decode, compression/decompression and/or encryptiontechniques may be used to facilitate the transmission of the compositemessage.

There is a possibility that the application server 112 only providesservices to users who have subscribed to such services. In this case, auser may be prompted to log in or sign up when he or she requestsservice from the server application 112.

The server application 112 may receive composite messages or servicerequests from a plurality of client applications 102 at the same time orat least during overlapping periods. It is therefore, beneficial thatthe server application 112 runs on a multi-user operating system thatenables multiple concurrent users to run applications in separate,protected sessions.

It should be recognized that FIG. 6 illustrates only one example of theclient application process 406 described with reference to FIG. 4. Otherimplementations are also possible. As an example, the client application102 may obtain document objects, output device objects and optional jobobjects in any combination of order or sequence, or may obtain themconcurrently. As another example, step 610 (transmitting compositemessage) may proceed before step 608 (creating composite message) isfinished. In other words, during the process when client application 102is obtaining objects from the user, output device 106, or otherapplications in the same information apparatus 100, the clientapplication 102 may also communicate with the server application 112concurrently or in alternating times to transmit partially or completelythe objects it has obtained.

FIG. 7 is a flow diagram of an exemplary server application process 701.Typically, an application server 110 is a much more powerful computingdevice than a mobile information apparatus 100. The server 110 maycontain a plurality of applications to interpret, process, or rasterizedigital documents into output data. Therefore, in one implementation thecore computation of an output process is executed in the applicationserver 110. The server application process 701 may include or utilize:

-   -   A server application 112 that receives a composite message        (including an output device object and a document object) from        client application102.    -   A server application 112 that processes the objects received to        generate device-dependent output data acceptable to one or more        output devices 106 selected by a user. The server application        112 may perform the processing function independently or in        combination with other applications.    -   A server application 112 that transmits output data back to the        information apparatus 100 requesting output service or services.

In step 700, server application 112 receives a composite message fromclient application 102. As mentioned earlier, the composite message mayinclude one or more of a document object, an output device object and anoptional job object.

After receiving the composite message from client application 102, theserver application 112 may process the document object and convert itinto output data. The output data generated is preferred to be in aformat or language acceptable or compatible to the output device ordevices 106 selected by the user.

In some cases, the document object or output device object in acomposite message may include only references, pointers and orinstructions. Such a configuration may, for example, reduce the size ofthe composite message and therefore make it easier to transmit through anetwork. In another case, the document object or output device object ina composite message may include missing or incomplete information.Digital document (e.g., output content) or output device parameters maynot be available or completely available in the document object or theoutput device object, respectively. In these cases, as shown in step704, the server application 112 may need to find or fetch partially orentirely the output content or device parameters from network nodes orotherwise obtain the content or parameters according to the pointers,references or instructions provided by the objects. For example, anoutput device object received in a composite message may containinsufficient output device parameters—only the brand name and modelnumber of a selected output device 106 may be provided. In this case,the server application 112 may either find or fetch the missing outputdevice parameters according to pointers or references provided in theoutput device object, or the server application 112 may consult atranslation service or application in order to obtain the completeoutput device parameters based on the partial or limited deviceinformation that is available. If no further information or data can befound or obtained, predefined defaults may be used to fill in missinginformation about the output device 106 or alternatively a warning or anerror message may be provided to the user for either continuation orcancellation of the process.

Step 704 (obtaining output content and or device parameters) may beentirely or partially skipped if the output content and output deviceparameters are already included in the document object and output deviceobject, respectively.

In order to process a digital document into device dependent outputdata, step 706 shows that the server application 112 may need to involveone or more helper applications. A helper application is any applicationin the server 110 or other network node that participates, helps orassists in the output process of the present invention. Examples ofhelper applications may include, software components, softwareapplications, device drivers, printer drivers, etc. The helperapplication or applications may perform one or more of the followingtasks among others:

-   -   Open, parse, or interpret a particular digital document format        or language.    -   Convert a digital document into an intermediate format, language        or data.    -   Process an input digital document into a raster format        including, as an example, one or more raster image processing        operations such rasterization, scaling, color correction, color        matching, segmentation, halftoning, compression, decompression        etc.    -   Convert or encode rasterized data into a device specific output        data.

The helper applications may participate in the output process in variousways. As an example, a server application 112 may involve one or morehelper applications to decode, parse, interpret, and or process adigital document into an intermediate file, format, language or data.And then the server application 112 may involve one or more helperapplications to further convert the intermediate result into outputdata. In another example, the server application 112 may involve two ormore groups of helper applications. The first group of helperapplications may decode, interpret, and process a digital document intointermediate file, format, language or data. The second group of helperapplications may convert the intermediate results into output data. Agroup may consist of one or more applications.

The server application 112 may identify, locate, invoke, launch, and orintegrate the appropriate helper applications or its components based onthe information (e.g., output device parameters) provided in an outputdevice object. A translation service (e.g., database application,directory service) may provide information to the server application 112as to where to find, obtain, or use the helper applications. The helperapplications may reside locally in the application server 110 orremotely in another network node. If a helper application is located inanother network node, the server application 112 may acquire or downloadthe helper application to the application server 110. Alternatively, theserver application 112, considering application availability or loadbalancing, may direct the digital document to be processed in anotherserver node where the helper application or applications are run.

In some cases, however, the server application 112 may have the abilityto process and generate the correct output data without involvingadditional helper applications and step 706 may be skipped. As anexample, the server application 112 may already contain softwarecomponents and feature sets capable of supporting and generating,different print data or output data formats or language such asPostScript or PCL or XML etc. Therefore, if the output device objectindicates that the output device 106 is a PostScript printer or anyother that it supports, the server application 112 may not need toinvoke, or download or incorporate a helper application or a helperapplication component such as a PostScript driver. The serverapplication 112 may just proceed and convert the digital document into aPostScript file as print data. PostScript is used here as an example,other formats, languages, data used for other types of print data oroutput data follows the same fashion.

When fetching output content or device parameters or software componentsor drivers from other network nodes, the server application 112 may needauthentication information in order to gain access. Such authenticationinformation may be already stored in the server application 112 (e.g.,in a user profile), in the received objects (e.g., job object, outputdevice object or document object), or the user may input authenticationinformation as needed.

While processing digital content in step 708, the server application 112may also distribute or send a presentation or GUI to a client device(e.g., information apparatus 100) to inform the user of the processingstatus (shown in step 709). The presentation or GUI in the client devicemay also capture user's preferences and inputs such as logininformation, security information among other preferences or inputs.This client/server implementation described here may provide anefficient computing environment. For example, logic for the serverapplication 112 may run in the application server 110 and itsdistributed user interface may run in the client device (e.g.,information apparatus 100). Therefore, only data relating to keystrokes,mouse clicks and screen updates may, as an example, travels through thenetwork. This client/server implementation may reduce bandwidthrequirements. Such a GUI may be implemented by, for example, instructingthe client application 102 to launch on the display screen of the clientinformation apparatus 100 a plurality of windows emulating the userinterface of the server application 112. The presentation of the serverapplication interface may be displayed efficiently through these windowsin the client application 102.

If the server application 112 receives no job object or an incompletejob object in a composite message, the server application 112 may assumedefault values for the job object or optionally, as shown in step 709,it may also launch a GUI in the information apparatus 100 as describedabove to obtain partially or entirely the job object fields (jobpreference) from the user. Information provided in the job object may beused in the process of generating output data (step 708).

It is noted that the output data (or print data in the case of aprinter) generated by the server application 112 may or may not be thefinal output data or the final print data that can feed directly into anoutput engine or a printer engine 1008. The output data sent to outputdevice 106 through information apparatus 100 may in some cases needfurther processing by an application in the information apparatus 100 orby the output controller 104 or printer controller 1010 or combinationbefore finally being sent to output engine or printer engine 1008. Allthese different combinations of processes and tasks or loaddistributions are possible implementations that shall fall within thescope and spirit of the present invention.

In step 710, the server application 112 transmits output data to theinformation apparatus 100 through network 108. Encryption techniques maybe applied to the output data to ensure security.Compression/decompression, encode/decode may also be implemented tofacilitate the transmission of output data over network 108.

FIG. 8 is a flow diagram of an exemplary final output process 801 forpervasive output, which may include or utilize:

-   -   A server application 112 transmits output data to information        apparatus 100.    -   An information apparatus 100 transmits output data, with or        without further processing, to the output device 106 selected by        the user.    -   An output device 106 generates final output based on the input        output data with or without further processing.

After receiving output data from server application 112, the informationapparatus 100 may then transmit the output data to the output device 106selected by the user in step 806. In some cases, the client application102 may further process the output data (as shown in step 804). Thisfurther processing in the information apparatus 100 may include one ormore operations such as encoding, decoding, compression, decompression,rasterization, scaling, color correction, halftoning, watermarking,adding templates, name, time stamps, etc., as shown in step 804, beforesending the output data to the output device 106 through communicationlink 116 in step 806. The client application 102 may involve otherapplications 103 for part or all of the processing in step 804.

During the time when server application 112 processes a compositemessage, the communication link 116 between information apparatus 100and output device 106 selected by the user may be constantly maintainedor it may be temporarily dropped and then resumed for transmittingoutput data. Encryption techniques may be applied to the output data toensure security.

Compression/decompression, encode/decode may also be implemented tofacilitate the transmission of output data over communication link 116.

After receiving output data from information apparatus 100, an outputdevice 106 may simply buffer the output data, as shown in step 810,before sending the data to output engine (or printer engine in the caseof a printer) 1008 for final output. In cases where an output device 106includes a printer controller 1010, output controller 104 or combinedcontroller, such controller may, jointly or individually, furtherprocess the output data (shown in step 814) before sending the data tooutput engine or printer engine 1008 for final output.

It is possible that a plurality of information apparatuses 100 maytransmit output data to an output device 106 at the same time or atleast during overlapping periods. It may then become necessary toimplement queuing or spooling functionality in the output device 106.Preferably, the output device 106 stores output data it receives in aqueue until the output data can be processed. Such queuing and spoolingcapability may be implemented partially or entirely by the outputcontroller 104 as an optional feature set. A user may be notified whenthe output device 106 is ready to process a particular output job or thestatus of the output job in the queue. Once the output device 106completes the output service, the user may terminate the output processby indicating such an intention through an interface on informationapparatus 100 or by moving the information apparatus 100 out of thecommunication range of the output device 106.

FIGS. 9A-9F show a series of graphical user interfaces (GUIs) renderedat different times on a display screen 901 of an information apparatus100 to illustrate one implementation of the output process 401 describedwith reference to FIG. 4. The illustrated example of user interface canbe implemented in an information apparatus 100 with or without anoperating system. The user interface is illustrated here only as anexample, other user interfaces may also be implemented and in no wayrestrict the scope and claims of the present invention.

Referring to FIG. 9A, screen 901 may be a part of an informationapparatus 100 such as, without limitation, palmtop, PDA, mobile phone,pager, Internet appliance, web pads, e-book etc. In this particularexample, as shown in screen 901, a user is reading a news articlepublished on the Internet using a document browsing application ininformation apparatus 100. Part of the article (e.g., photo 900) cannotbe fully displayed and so is indicated by a link or tag. The inabilityto fully display photo 900 may arise from a variety of reasons includinglimitations in display capacity, memory, processing power, or lack ofsupport by a service provider, application, communication protocols, orsoftware protocols, among other reasons.

As shown in FIG. 9A, a document browsing application in the informationapparatus 100 may have a plurality of functions and feature setsavailable to the user through the GUI. A printing function control 902may be one of these features. The output or printing functionality maybe provided by accessing a client application 102 with or without anoperating system. In this illustrated example, the user may invoke theoutput function by selecting the print function control or icon 902. Theuser can make the selection by using, for example, any of a keyboard,keypad, mouse, stylus, soft keys, push buttons, software command, touchsensitive screen, voice-activated command, among others. Other displaymethods or functionalities or feature sets such as a drop down list, apull down menu, among others, are also possible.

When the user selects Print function control 902, the client application102 may be invoked or launched. The output process 401 described withreference to FIG. 4 may then proceed. The client application 102 mayobtain a document object (e.g., the news article and or the pointer ofthe new article) from the document browsing application. As describedearlier, the user may be able to choose to output the reduced digitaldocument (e.g., without photo 900) as displayed in the informationapparatus 100 or to output the full digital document in its originalform (e.g., with photo 900). We assume for this example that the userhas preset the default to always output the full digital document in itsoriginal form.

Since the user does not have a pre-selected or default output device106, a discovery process 520 may automatically proceed in this example.A communication manager may coordinate the discovery process 520. Asdescribed earlier, the communication manager may be part of the clientapplication 102 or may be a separate application that can communicatewith the client application 102.

Assuming in this example that he or she did not specify any searchingcriteria for the discovery process 520, the user may be presented withall the printers 106, if any, that are available to take the print job.In the case where no available printers are found (not shown here), theuser may also be notified and provided with alternatives as describedearlier in the discovery process 520.

Assume in this example that three printers 106 are found. Theiridentities and service charges are listed on the GUI illustrated in FIG.9B. The user may select a “more information” control or icon 904 to getmore detailed information on each available output device 106. The usermay select a printer 106 from the list, for example, by moving a cursorto highlight the name of the selected printer 106 and then selecting a“select” control or icon 906. The user may also withdraw the servicerequest from any of the three printers by selecting a “cancel” controlor icon 907.

Assume in this example that the user has selected the PH inkjet 260printer. The next GUI, as illustrated in FIG. 9C, provides the user witha confirmation. Information included in the confirmation may include (1)address (pointers) of the digital document intended for output, (2)printer selected, (3) service fee that will be charged. Various paymenttypes and methods may be used. The user may also choose to providepayment, deposit or escrow in various forms. However, for simplicity ofillustration, only two are shown in this particular example. If paymentby credit card is selected, the user may be prompted to provide creditcard information. In one example, the user may then provide suchinformation through a GUI (not shown) on the information apparatus 100,which then transmits the information to the selected output device 106.In another example, the user may provide credit card informationdirectly to the selected output device 106 through an interface provideby the output device 106. If the user has selected the cash paymentoption, cash may be physically deposited into a cash collector installedor connected to the selected output device 106. Or the user may transfercash in electronic form from the information apparatus 100 to the outputdevice 106. The user may or may not need to manually input paymentinformation such as a credit card number. A user profile and or avariety of payment information may have already been saved in the user'sinformation apparatus 100. The client application 102 may automaticallyfetch necessary payment information and process the transaction with theoutput device 106 or with a remote transaction server. If the outputservice is free, no payment information will need to be collected. Ifthe output service is a subscription, then membership, login,authorization, or security information may be collected instead or inaddition.

After the user confirms all the information and provides payment ordeposit as requested, the user may be prompted to describe the jobobject as illustrated in FIG. 9D. Some of the selections for the jobobject input illustrated here may be device-dependent while others maybe device-independent. The user can make selections by using, forexample, radio buttons among other controls. Some default values, suchas items 908 and 909, may have been preset by the user or automaticallycalculated by the client application 102 after negotiation with theoutput device 106. The user may also change all or some default values.After describing the job object, the user may select the “Next” controlor icon 910 to move to the next step. It should be noted that only someexemplary or commonly used selections are displayed in the exemplary GUIof FIG. 9D. The user may make further selections by, for example,selecting the “more option” control or icon 911 or by simply scrollingdown the display.

After the user submits the job object, the client application 102 sendsa composite message 430 to an application server 110. The compositemessage 430 may include a document object, a printer object, and a jobobject. The application server 110 may contain server application 112.In this example, the server application 112 may need to fetch the outputcontent (in its original format) based on the pointers provided in thedocument object. The server application 112 may obtain or invoke helperapplications to process the output content (e.g., the news article) intoprint data that can be accepted by the printer 106 selected by the user(e.g., PH Inkjet 260). The server application 112 and or the helperapplication may perform one or more raster image processing operationson the output content. During this server application process 701, thesever application 112 may optionally launch a GUI in a window 912 asshown in FIG. 9E to provide the user with estimated processing time andor status. The user may choose to hide this window by selecting the“hide” control or icon 914. The user may also choose to abort theprocess by selecting the “cancel” control or icon 913.

After it has finished processing the output content and has generatedthe necessary print data, the server application 112 transmits the printdata to the selected output device 106 through the user's informationapparatus 100. Transmission may be wired or wireless. The clientapplication 102 in the information apparatus 100 may or may not furtherprocess the print data before passing it to the output device 106. Theoutput device 106, after receiving the print data, may queue the printjob in its memory or storage component until it is ready to process thisparticular print job. As shown in FIG. 9F, the user may be notified whenthe original document is being printed or has been printed out. The usermay also be notified of the service fee that has been, for example,charged to a credit card or deducted from a cash deposit. The user maybe provided with a “more option” control or icon 916. For example, theuser may choose this printer as a preferred default printer so that forthe next output or print operation a discovery process 520 may beskipped or shortened if this printer 106 is found to be available.

FIGS. 9A-9F provides only one example of output process 401 describedwith reference to FIG. 4. While the above descriptions contain manyspecificities, these should not be construed as limitations on the scopeof the invention, but rather as exemplary embodiment thereof. Other userinterfaces and embodiments may also be implemented. The process may alsobe implemented with more or fewer steps. As an example, securityverification and authentication may be added to the steps illustrated inFIGS. 9A-9F. As another example, the step illustrated by FIG. 9B may beskipped as the user may have pre-selected or preferred a defaultprinter.

These and other possible variations do not however depart from the scopeand spirit of present invention.

Having described and illustrated the principles of our invention withreference to an illustrated embodiment, it will be recognized that theillustrated embodiment can be modified in arrangement and detail withoutdeparting from such principles. In view of the many possible embodimentsto which the principles of our invention may be applied, it should berecognized that the detailed embodiments are illustrative only andshould not be taken as limiting the scope of our invention. Rather, Iclaim as my invention all such embodiments as may come within the scopeand spirit of the following claims and equivalents thereto.

What is claimed is:
 1. An information apparatus for receiving outputdata from one or more servers over a network, the network includes theInternet, the output data includes image content, picture content, audiocontent, or video content, individually or in any combination, theinformation apparatus comprising: one or more processors; memory orstorage; software or firmware stored at least in part in the memory orstorage; a digital capturing device for capturing images or forcapturing video; a display screen that is at least an output device foroutputting or rendering content; one or more wireless communicationunits that include wireless communication circuitry for wirelesscommunication, the wireless communication includes wireless local areanetwork communication; wherein upon execution of the software or thefirmware by the one or more processors, the information apparatus: (1)couples, using the wireless communication circuitry of the informationapparatus, the information apparatus to a wireless local area network;(2) establishes, using the wireless communication circuitry of theinformation apparatus and via wireless local area network, a wirelessconnection to the one or more servers over the network that includes theInternet; (3) wirelessly accesses, using the wireless communicationcircuitry and via the wireless connection, services provided by the oneor more servers, the wireless accessing of the services is based, atleast in part, on having appropriate authentication information orsecurity information for accessing the services provided by the one ormore servers over the network that includes the Internet; (4) captures,using the digital capturing device, digital content, the digital contentincludes one or more of a picture, an image or a video, individually orin any combination; (5) obtains, at the information apparatus, a contentobject, the content object includes the at least part of the digitalcontent; (6) wirelessly transmits, using the wireless communicationcircuitry of the information apparatus, the content object from theinformation apparatus to the one or more servers over the network thatincludes the Internet; (7) wirelessly receives, using the wirelesscommunication circuitry of the information apparatus, output data fromthe one or more servers over the network, the wireless receiving of theoutput data being in response to having transmitted the content object,the output data has a relationship, at least in part, to the contentobject that is transmitted to the one or more servers; (8) renders oroutputs or plays, at the display screen output device, at least part ofreceived output data; the output data includes at least one of an imagecontent, picture content, or video content, individually or in anycombination.